Routing notes Sep 2, 2026 6 min read
What a model router should never do
Three rules we wrote before the first line of code, and the one we broke anyway.
Ines Calloway
Co-founder
A router sits between your product and every model you pay for. That position is powerful, and power in the request path is dangerous. Before we wrote any code we wrote down what Prism would never do, and taped the list above the desk.
Never route silently
Every decision the router makes is written to the log with its reason: which policy fired, which models were considered, why the winner won, and what it cost. If a fallback fired, the log says so and says why. A router you cannot audit is a router you cannot put in front of paying customers.
Never take a cut per token
The moment a router earns more when you spend more, its incentives point the wrong way. Prism charges a flat platform fee and passes provider prices through at list. When we find you a model that is six times cheaper, we make exactly the same money as before.
Never change the endpoint
The API is OpenAI-compatible and stays that way. We change what is behind the endpoint — new models, new providers, better scoring — never the shape your code depends on.
The rule we broke
We had a fourth: never re-order a response. Then we shipped streaming fallbacks, where a second provider picks up a stream the first one dropped mid-sentence. The client sees one stream; the log sees two. It is the right trade, and it is the only place the router does anything you did not literally ask for. We wrote it down, too.